This invention relates to an image processor suitably used to decode image information.
The compression and decompression of moving picture data is internationally standardized by MPEG (Moving Picture Image Coding Experts Group) named after the working group of the ISO/IEC. A major element of an MPEG decoder used to reproduce moving picture data is a data processing unit that is formed of a variable length decoder (VLD), an inverse quantizer (IQ), an inverse discrete cosine transformer (IDCT), and a motion compensator (MC). Such an MPEG decoder further requires a memory for a plurality of frames for motion compensation and interlace conversion.
One of the outstanding features of the MPEG is that two frames (i.e., an earlier frame and a later frame in time) are used for motion compensation reference. However, the application of motion compensation to every image produces propagation of errors, and problems with special reproduction. To cope with these drawbacks, an intra-coded picture (xe2x80x9cI-picturexe2x80x9d), a predictive-coded picture (xe2x80x9cP-picturexe2x80x9d), and a bidirectionally predictive-coded picture (xe2x80x9cB-picturexe2x80x9d), have been introduced. An I-picture, i.e., a picture of the coding type I, does not refer to any other images at all. A P-picture, i.e., a picture of the coding type P, is subjected to motion compensation by an earlier frame. A B-picture, i.e., a picture of the coding type B, is subjected to bidirectional motion compensation by both an earlier frame and a later frame. B-pictures are not used as a reference frame when other frames are decoded.
A coding type prediction is now described. A bit stream of input pictures is entered to an MPEG decoder in the order I0, P3, B1, B2 for decoding. The picture P3 is motion compensated from the picture I0. The picture B1 is motion compensated from the pictures I0 and P3. The picture B2 is motion compensated from the pictures I0 and P3. The display of the pictures is made in the order I0, B1, B2, P3. The order of decoding and the order of display disagree and it is therefore necessary to make a change in the order. Additionally, decoding of the pictures B1 and B2 requires data for two image frames of the pictures I0 and P3, therefore requiring a frame memory for two image frames as a reference to motion compensation. For this reason, the MPEG decoder requires two frames as a reference to motion compensation.
The order of decoding by the pixel and the order of output by the pixel of the MPEG are described. In the television system, all the even-numbered lines are provided prior to the odd-numbered lines, in other words pixel output is performed every other line, starting with the upper left and ending with the lower right. An area made up of the even-numbered lines is called the top field while on the other hand an area made up of the odd-numbered lines is called as the bottom field. In the interlace output, a top field is first provided from the upper left down to the lower right and a bottom field is then provided from the upper left down to the lower right.
Image data are 2-D data, and data adjacent in space to each other are considered having a strong correlation therebetween. However, for the case of interlace output, paying attention to a specific top field""s line, its overlying line, i.e., a line one line up of the top field line in question, is a bottom field""s line. Although pixels of a line have a very strong spatial correlation with pixels of its adjacent line, they are separated in time. Accordingly, when motion is very active, there may be the possibility that a line has a stronger correlation not with a line one line up but with a line two lines up, for they lie adjacent to each other in time. Assuming that such a possibility may exist, the order of decoding by the pixel of the MPEG is roughly divided into two types, namely the one for the frame structure and the other for the field structure.
Under the MPEG standard, decoding is carried out using 16 by 16 pixels as a basic unit (called a macroblock). The macroblock is decoded from left to right. Here, the rightmost pixel of the uppermost line of an image is contained in the rightmost macroblock of the image. When the decoding of the rightmost macroblock is completed, this means that data for 16 lines are decoded. Accordingly, the decoding of each of data for 16 lines is almost simultaneously completed.
For the case of the frame structure, data for one image frame constitute a macroblock formed of 16 by 16 pixels, and decoding is performed per macroblock. As a result, top and bottom fields are almost simultaneously decoded. This produces a complete disagreement between the order of decoding and the order of image output. A change in the order is required to make.
For the case of the field structure, one image frame is divided into two sections, a top field and a bottom field. Each field includes macroblocks, each of the macroblocks being formed of 16xc3x9716 pixels, and decoding is carried out per macroblock. A single macroblock does not extend over both the top and bottom fields. After all the data for the top field are decoded, the data for the bottom field are subjected to decoding. In this case, the images will be provided in almost the same order that they were decoded. However, the order of decoding and the order of image output do not agree perfectly because of decoding by the macroblock.
Image output is performed such that the top field of an image is first provided and the bottom field is then provided. When paying attention to the time point at which the output of the last eight lines of a top field starts, the decoding of a macroblock for the last 16 lines of the image frame must be completed before the starting point. The reason is that it is not until the decoding of the last macroblock of an image starts that the values of the rightmost 16 pixels of the last eight lines are determined. Therefore, both the top field decoding and the bottom field decoding must be completed at the foregoing point. From the point on, the eight top field lines and all the lines of the bottom field must be provided sequentially; however, the data for them have already been decoded. Therefore, if the data are not stored in frame memory, the data will be lost before they are output, therefore resulting in no image output. This gives rise to a need for the provision of a frame memory with an about half-frame storage capacity to store all the data for the bottom field and the eight top field lines.
To sum up, a 2-frame memory is required for motion compensation, and an about half-frame memory is required for B-picture interlace conversion, in other words at least an about 2.5-frame memory is required.
Ishiwata et al. report, in a paper entitled xe2x80x9cDevelopment of an MPEG2 Decoder LSIxe2x80x94Efficient Memory Allocation,xe2x80x9d Proceedings of the 1994 IEICE Spring Conference, C-659, March ""94, an MPEG decoder that employs a 1.5-frame memory for B-picture interlace conversion. Takabatake et al. report, in a paper entitled xe2x80x9cDRAM Interface for MPEG2 Video Decoder LSI,xe2x80x9d Proceedings of the 1995 IEICE General Conference, C-586, March ""95, an MPEG decoder that employs a one-frame memory for B-picture interlace conversion.
However, both the above-described MPEG decoders are expensive because they require a memory with a storage capacity ranging from one frame to one and a half frames, for B-picture interlace conversion. In principle, the B-picture interlace conversion requires just a half-frame memory and therefore there still exists room for a reduction of the required memory storage capacity for B-picture interlace conversion.
Accordingly, it is an object of the present invention to achieve a reduction of the storage capacity of a frame memory employed in MPEG decoders. It is another object of the present invention to achieve efficient use of the frame memory.
The present invention is based on the following points. The first point is that once a B-picture is provided it will not be reused. The second point is that it is possible to predict when the region of a macroblock currently being decoded is provided by analyzing an additional information portion (called xe2x80x9cheaderxe2x80x9d) of an input picture.
The present invention provides an improved image processor with a frame memory wherein (a) the frame memory includes five blocks, each of the five blocks having a storage capacity to store a half image-frame, and a single additional block having a smaller storage capacity than each of the five blocks, (b) four blocks, selected from among said five blocks, serve to temporarily store I- and/or P-pictures for motion compensation reference and the remaining one block and said additional block together form a data memory for B-picture interlace conversion, and (c) thereafter, the data memory is reconstructed using one of the selected four blocks and the additional block.
Such arrangement makes it possible to achieve, with an about 2.5-frame memory, not only B-picture interlace conversion but also motion compensation with respect to all types of pictures. Additionally, for the case of the three or more successive I- and/or P-pictures, the data memory is not used for B-pictures, which produces the advantage that one block, which forms a portion of the data memory, may be used for storage of I- and/or P-pictures.
In the above-described image processor, it is preferred that (a) the five blocks and the additionally block are divided into respective pluralities of slots each of the slots having a predetermined storage capacity, (b) the image processor further comprises a slot control memory for storing slot numbers and a controller for controlling both read and write operations of the data memory by making use of the slot numbers stored in the slot control memory, and (c) the controller writes a slot number, which was used in a write operation of entering information to the data memory, to the slot control memory for performing a read operation of acquiring information from said data memory.
As a result of such arrangement, it becomes possible to update the contents of the slot control memory at the time of a write operation of the data memory in such a way so as to allow the data memory to be read in a correct slot order. Accordingly, it is possible to achieve B-picture interlace conversion with an about half-frame memory by the arrangement that the operation of write to a slot of the data memory is carried out immediately the operation of read from the slot is carried out.